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In the academic imagination, depth and attention are the highest of virtues. But in pushing students to apply high-attention strategies to 


all incoming information, we risk creating a new and dangerous shallowness. 
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In this essay, I will talk (eventually) about information literacy, misinformation, and the fallibility of human reason. But, if you'll indulge 
me just a bit, I want to talk about real-world decision-making first. In looking at the structure of decisions we can better illuminate how 
the information-seeking process works when up against constraints — and it’s an information literacy that is honest about constraints 


that I argue we must embrace. 


So let’s start with this: Iam currently in the middle of a job transition. My wife and I have begun the process of looking for an affordable 
neighborhood. But what is affordable? Our current house was bought with different salary assumptions, but the housing market has 
shifted. For the first time in our lives, we actually have accrued some value in our current house that allows for a substantial down 
payment. On the less positive side, we’re moving into an expensive market where even entry-level housing is going to put a bigger hole 
in our budget than we are used to. The house prices are, frankly, a bit shocking. We’re going to have to spend more, but are unsure how 


far we can push our housing budget. 
What do we do to figure out the maximum amount we can spend? 


One approach might be to list out all of our income and all of our expenses. We could then decide which expenses we could cut, where 
we could consolidate items, and plot likely future earnings and required savings. Get the spreadsheet out. We could really throw 
ourselves into it — what is the difference in grocery costs in our new neighborhood? How much is the house likely to appreciate in 


value? 


Or we could do what most people do: Start with the rule that in the United States one should not spend more than 28% of gross income 
on a house. From there, if there is anything particularly unusual about the house (requires extensive repairs) or us (two kids in college at 


the moment) we can nudge that percentage down a bit. We’d start not with a complex spreadsheet, but with a simple rule. 
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In the Face of Complexity, Less Can Be More 


As you're probably aware, financial experts recommend the second course of action. Many people assume that recommendation is 
based on the lack of skill of the average homeowner — that is, that the recommendation is for those that don’t know how to work a 
spreadsheet, or how to research likely home appreciation rates. The rule is fine for the rookies, people think, but to make a really good 


decision requires several orders of magnitude more research, background knowledge, and study. 


But this is not the case. The rule is there because the complexity of the decision makes a farce of pursuing detail. Yes, I can plot out my 
current expenses to the last penny, and perhaps I would find I can spend much more than anticipated on a house. But what about 
unexpected future expenses? Likewise, I can calculate our current spending, but levels of spending are relative in part to the amount of 


disposable income a family has. 


Add to this what we’ve all observed in ourselves or others when we’re just dying to make a purchase that is out of our budget: The more 
calculations in our “can I afford this?” calculus, the more opportunities there are for us to unwittingly put a thumb on the scale. We often 
don’t know we're doing it: The house or apartment is a bit closer to work, so surely that will save us $40 a month (rounded up to $50), 
and with that kitchen we’re going to eat in more often, which is practically money in the bank, right? Never mind that every calculation 


is an opportunity for error. 


Because we're fallible beings, we start with a rule that accepts we don’t know the future and tend to distort the present. Researcher Gerd 
Gigerenzer, a psychologist, has long tracked how these simple rules have improved decision-making in a variety of professional 
contexts. From estimating the amount of fuel needed for a flight to improving emergency room triage, he has termed this approach 
“rationality for mortals.” He has demonstrated the wide variety of situations where more information, more calculation, and more 


precision makes us substantially worse at bottom-line decision-making compared to simpler processes and rules of thumb. 


Why do we continue to assume that more is more, then? In his 2008 book, Rationality for Mortals: Risk and Rules of Thumb, Gigerenzer 
discusses the ways in which the Enlightenment modeled the rational decision maker as a sort of “secularized God.” In such a scheme the 


ideal decision maker benefits from: 
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- Omniscience: They can know or find all variables that would impact a decision 
- Omnipotence: They have unlimited time, mental resources, and intellectual capability to come to the best decision 


- Determinism: Given all relevant variables the world is deterministic enough that the future is foreseeable 


If you take this view, it’s true: Each new fact, statistic, or spreadsheet cell gets us closer to omniscience. Each minute we spend produces 
an incrementally better decision. And none of that will be undone by things unforeseen (or influenced by the bias of the decision- 


maker). 


I’m willing to stipulate that for a researcher, working in their field and investigating a narrow issue, such an ideal might be useful. Given 


more time and more information, you might see better results. 


But what about the rest of us? What happens to specialists when they are outside their domain of expertise? And how might such non- 


expert needs inform how we teach? 


Information Literacy for Mortals 


A few years ago, I developed the SIFT method that makes use of quick checks over deeper analysis. Its fundamental insight is that when 
we rush to deeper analysis before assessing the basics, we end up making poor judgments about information. Just as we might want to 
start assessing the house we can afford by looking at its monthly cost as a percentage of gross income (rather than computing an all- 
encompassing budget), we encourage students to start with a few simple moves. Before diving into an article and asking the 26 questions 
and subquestions of CRAAP, we encourage them to check out the source’s page in Wikipedia. Does anything on that page surprise them? 


Is the source what they expected it was? If not, their assumptions about what they’re looking at may need adjusting before they proceed. 


Before addressing the complex question of whether they think a news story is plausible, they are encouraged to ask a simpler question: 
Are other reliable outlets reporting it? Again, if it’s the sort of story that would be reported widely and there aren’t other outlets jumping 
on it, maybe the story isn’t what they thought it was. The approach has proven quite effective, with a recent study showing dramatic 


increases in student capability to reach competent judgments about sources and claims after completing a short module. 


I’ve talked extensively about SIFT (and the lateral reading research that informs it) over the past several years. But I seldom get to talk in 
detail about how weird classroom activities are when you take a closer look. In classroom information literacy sessions over the past few 
years, I noticed that given a bit of time on a question — “Did this thing happen?” “Is this source reliable?” — students do well. They 


apply the SIFT method we’ve taught them and make decent judgments about claims and sources. 


Given a lot of time, however, students often do worse. And, as you watch the individual groups, you begin to understand why. Early on 
in their research, the students discover the fundamentals: Source X is from an oil industry lobby group, source Y from a respectable 


trade organization. But students keep going, engaging in a process I call search result overfitting. 
What does this look like? 


As the search progresses past broad strokes and easy answers, the students find more information with a smaller scale impact. They 
discover the oil industry group did produce a well-cited report a few years ago. They find that a reliable publication was called out for an 
inaccurate story several years back. It also was founded in 1908 as a Republican paper, which seems interesting. They probe a respected 


health site where someone who used to sit on the board is also on the board of a company recently charged with financial malfeasance. 


In statistics and machine learning, “overfitting” is a phenomenon where too many data points of varying usefulness are incorporated in 
a model. Because the model does not distinguish between “signal” and “noise,” a model is produced that incorporates the available data 
better than alternative models and yet turns out to be less useful than a model that incorporates only a few data points. And to some 
extent this is what is happening here. There is a belief from the students that every fact discovered will play some part in their analysis, 
and, in practice, the later a fact is discovered in a search, the more weight they seem to give it. This, of course, is opposed to how facts 


often present themselves, with the most important facts emerging early in a search, and the least relevant facts emerging later. 
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We see the same features in our post-tests. Our recent research has shown dramatic increases in student capability when they learn basic 
lateral reading techniques. A student using basic lateral reading techniques might note that a Wikipedia check of a specific cancer 
therapy revealed that it is long debunked and its inventor lost their license for practicing it. But they'll also point out that the site is 
running a lot of advertisements, which seems suspicious. This is the equivalent of noting that you would not recommend taking a ride 
from Jeffrey Dahmer as he is both a serial killer and also chronically late. It’s a good decision, but there’s clearly still confusion about the 


relative importance of criteria. 


And, mind you, these are the students who do well! For many others, the result is a feeling of being increasingly overwhelmed, as each 
link on each page provides yet another fact to slot somewhere in the student’s working memory, as they try to accurately weight the 
importance of each thing they discover. The biggest struggle students have with SIFT is not with the skills themselves. Rather, their 
biggest struggle is with their underlying assumption that the quick answer is a bad answer. It seems a matter of honor more than 


anything else: Sure, you can get an accurate answer quickly with the right skills — but how worthy of admiration is that? 


Something is seriously wrong here. At the very least, it’s clear that the strategies students are using in our classes are not well matched to 


the task environment. 


Our students, after all, come to the web as people with limited time, limited domain expertise, and access to a near infinite amount of 
detail. And I’d argue that they apply to that environment the behaviors that we lionize in academia, and encourage them to perform: 
The application of deep attention, the discovery of endless tiny details, and a publish-or-perish related obsession to not only having a 


useful perspective, but having a unique one. 


This is a bad match. As Herbert Simon noted a half-century ago, you cannot solve the problem of a scarcity of attention by requiring 
more attention. Attention is, after all, the scarcity. To the extent that attention is scarce, you need to develop strategies and techniques 


that use less, not more of it. Encouraging students with a little expertise and less time to engage deeply is a recipe for disaster. 


Then, why do we do it? 


What is the Task Environment of Civic Information Literacy? 


I'd argue that there is a persistent belief in higher education that all thinking is just a degraded form of research. In this view, for people 
to reap the benefits of science they should become scientists. To think more effectively, people can learn philosophy. What is 
conveniently ignored is that research as practiced in higher education is not at the top of a pyramid of cognitive virtue. Instead, research 
reflects a set of practices grounded in a specific endeavor, namely the discovery of new knowledge and the testing of old knowledge by 


people with substantial expertise and access to a specialized discourse community. 


There are many ways in which the task environment of the current citizen is not only different, but orthogonal to such concerns. For 
instance, the environments in which most student citizens currently consume information are social feeds. In general, these are high- 
volume, low-attention environments used to sort through many different options for applying one’s deeper attention. They are not 
libraries as much as attention markets. In such an environment, the question for a citizen is never whether something is good enough to 
cite — they are often asking whether something is good enough to read, worth continued viewing, or perhaps just “plausible enough to 


worry about.” 


Additionally, when engaged in information-seeking (or, more often, information-encountering), the citizen is often not looking for 
academic precision, but to make good decisions under conditions of uncertainty. The question a person researching a vaccine must ask 
is not what the full inventory of vaccine side-effects might be, but whether the risks of taking it outweigh the risks of declining it, both 


on a personal and community level. 


When looking at a recent event, the citizen often wants to know the likely proximate cause, as they build a rough model of what to 


worry about and what to celebrate. Are ballots found in a ditch an indication of election fraud or something less sinister? Most of the 
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time, the question they have is even simpler: Can I ignore this for now? Or do I need to understand it? And if I do need to understand it, 


who should I trust to tell me more? 


Reputation Heuristics and Building Out From the Student 


Contrary to the belief that this process is a form of light-weight research, it bears far more connection to non-academic capabilities our 
students already bring into the classroom. After all, in their offline social interactions, students have at least some techniques to sort fact 
from fiction and hype from straight talk. They understand concepts like conflict of interest when it comes to rumors in their friend 
group or when it comes to whether to take a used car dealer’s word on the condition of a car. They know that it’s important to assess the 
reputation of the speaker when assessing a claim. They might not know a phrase like “epistemic trespassing,” but they would find the 


idea that they should trust an electrician's opinion on their house’s wiring more than a plumber’s unremarkable. 


Humans are social animals, and have developed over time complex and often accurate rules of thumb about when people can be trusted 
and when claims merit attention. And these rules often require far less information than more complex investigations. It’s clear those 
intuitions do not map directly onto the larger information space of the web. But rather than trying to map academic understandings of 
information onto non-academic tasks, it seems to me much more interesting to start here, with what the students already know and 
build on that. How can a set of social intuitions and rules that work quite well in a physical world of small groups map onto this new 


virtual environment? What skills do the students already have that can be adapted to this larger world? 


As an example, many people follow a simple rule in regards to resolving conflicting stories that circulate around their social circle: Start 
by finding people in a position to know the real story, then eliminate those who are untrustworthy or have a substantial conflict of 
interest. Listen to the most trustworthy person who remains. It’s not perfect, but it works, and it works by eliminating information in a 


way that would make a researcher wince. 


Now, on the web, a lot of these evaluation techniques fall apart, because the environment of the web is different from that of a small 
social group. In small social networks, for instance, people track who in their social network is reliable, who is not, who tries to play it 
straight, and who stirs up drama. Sources have histories; they earn their reputations over time. Likewise, if a fight breaks out and one is 
looking for a source, “position to know” can be as simple as “Who saw the fight?” But how do you assess who is in a position to know 


about climate change, or deficit spending? When thousands are in such a position, what do you make of disagreements between them? 


One approach to dealing with multiple conflicting sources might be to bring the students into a research process, to assume that a 
complex environment requires more complexity, more depth, and more data. Yet, as we saw with our budgeting example, depth does 
not seem to help. And this holds true in information literacy as well — the foundational study on the technique of lateral reading 


showed how lost accomplished researchers get on the web when they apply their research skills to an unfamiliar topic. 


A better way is to see that with some modifications the simple rules students use outside schoolwork can work. Yes, it’s true that we do 
not have the personal knowledge to make reputation judgments of novel sources online. But the web makes it easy for a reader to get a 
quick summary of the reputation of the source they are reading before they read it. So in SIFT, the methodology I teach, we highlight 
this problem of the web for the students (unknown reputation of sources) and show how simple moves like looking for a summary of an 
organization in Wikipedia can address that problem within seconds. Similarly, the common social heuristic that a story heard from 
multiple independent sources is more likely to be true can be turned upside down on the web, where repeated exposure to 
misinformation is common, and sources are often less independent than one realizes. Yet with some training, it can still be applied if 
approached intentionally: A quick search can reveal if the sort of people you’d expect to be reporting on a story are in fact reporting on 


it. 


These are small but important modifications. The fundamental idea driving them is that reputation of claims and sources is 


discoverable on the web, and discoverable relatively quickly. And what we find is that if students take steps to evaluate that reputation, 
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even briefly, before engaging with content, they end up being served by the off-web skills they have developed quite well. Their 


intuitions improve dramatically. 


Add in the small connective piece of a few simple web techniques and they are once again able to tap into abilities and understandings 
they already have. And to the extent SIFT and other lateral reading approaches have been successful (and they have been quite 
successful), a big part of the secret is this: Rather than seeing the students’ non-academic rules of thumb as a thing to be eliminated and 
replaced by a more logical framework, we see students’ native understanding of the social heuristics of information as capability to be 
tapped. They just need to know how to map those extant understandings and heuristics to the web, where reputation is stored less in 


one’s head than it is in the network. 


And as for us as teachers, as librarians, as decision makers? Our task is much more challenging. We have to understand that in the 
context of decision-making, simple can be good, less can be more, and the skills our students bring into the classroom may be a more 
valuable starting point than anything the traditional research process can provide. Some of that may come in adopting lateral reading 
approaches, such as SIFT, or the Stanford History Education Group’s COR framework. Some is in getting students to understand the 
social nature of knowledge and why quick judgments about the reputation of claims and sources can be more valuable than deep 


personal analysis of them. 
But the first and biggest step may be a simple shift in attitude. 


A few months ago, I was giving an introductory presentation on SIFT as a guest lecturer, and I walked through an example: A tweet 
about ankle monitor surveillance of unvaccinated students. It is a three-step example with a falsely contextualized claim that 
demonstrates SIFT quite well. Investigating the source reveals the publication has a history of spreading false information. Finding 
better coverage reveals that reliable publications told a very different story of the event. And clicking through to the source of the claim 
(“trace” in SIFT terminology) reveals even the linked article does not support the tweet. But I did something I have learned to do over 


time. After I revealed that the source of the tweet was associated with false stories in the past, I paused. 


“Now, I said, “You could actually stop here. You saw a tweet, got mad, hovered over the source and found out this wasn’t a great source. 
So maybe it’s not worth any more of your attention. Shut down the phone and see what’s on Netflix, or go outside and get some sun. 


You have no obligation to waste good attention on bad sources. But if you wanted to dig deeper...” 


And then I continued on, showing all the other ways you could approach it. At the end of the class, a number of students came up and 


asked various questions. One student was hanging back, not saying anything. 

“Was any of that useful?” I asked, self-effacingly, trying to draw her into the conversation. 

“Yeah,” she said. “That part where you said we could stop there? The Netflix thing? No one ever says that. It feels almost wrong.” 
“You think it’s wrong?” I asked. 

“Oh, no,” she said, “I agree! I’ve just never heard it in a classroom.” 


There’s a wealth of opportunities to pursue as we think about what “information literacy for mortals” might look like. But if we could 


start here, with that comment, we might make some progress. 


Read the author’s reflections on what inspired this essay 
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new directions that inform future research. Read more about the series » 
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Provocation Series Champions 


To see a list of organizations pledging support of the PIL Provocation Series, click here. 


About PIL 


Project Information Literacy (PIL) is a nonprofit research institute in the San Francisco Bay Area. Since 2009, in a series of 12 groundbreaking scholarly research studies, 
PIL has investigated how U.S. college students and recent graduates utilize research skills, information competencies, and strategies for completing course work, engaging 


with news, and solving information problems in their everyday lives and the workplace. 


Essays in the PIL Provocation Series are open access and licensed by Creative Commons. 
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PIL does not track your interactions with our site. 
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